Understanding Student Procrastination via Mixture Models 
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ABSTRACT 


Time management is crucial to success in online courses in 
which students can schedule their learning on a flexible basis. 
Procrastination is largely viewed as a failure of time manage- 
ment and has been linked to poorer outcomes for students. 
Past research has quantified the extent of students’ procras- 
tination by defining single measures directly from raw logs 
of student activity. In this work, we use a probabilistic mix- 
ture model to allow different types of behavioral patterns 
to naturally emerge from clickstream data and analyze the 
resulting patterns in the context of procrastination. More- 
over, we extend our analysis to include measures of student 
regularity—how consistent the procrastinating behaviors are— 
and construct a composite Time Management Score (TM). 
Our results show that mixture modeling is able to unveil 
latent types of behavior, each of which is associated with 
a level of procrastination and its regularity. Overall, stu- 
dents identified as non-procrastinators tend to perform sig- 
nificantly better. Within non-procrastinators, higher levels 
of regularity signify better performance, while this may be 
the opposite for procrastinators. 
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1. INTRODUCTION 


As colleges and universities continue to increase the num- 
ber of online course offerings, these classes are becoming a 
normal part of students’ learning experiences. While on- 
line courses have made learning more accessible to students, 
prior work suggests that students enrolled in online courses 
have worse learning outcomes when compared to students 
enrolled in face-to-face courses [2]. One important reason 
for this is that the online learning environment requires a 
higher degree of self-regulation than the face-to-face envi- 
ronment [5]. Students must effectively plan and regulate 
their learning time, and monitor their own progress in or- 
der to meet important deadlines [31], but students may lack 
some of these important skills. Moreover, online courses 
have a high degree of anonymity. Students are not phys- 
ically present in a classroom, and their activity on Learn- 
ing Management Systems (LMS) is not made public to the 
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rest of the class. This absence of face-to-face accountabil- 
ity may cause students to disengage with the course much 
more than they would in traditional classrooms. The lack of 
structure and anonymity may lead students to procrastinate, 
putting off work until close to important deadlines. There- 
fore, understanding students’ learning behaviors relating to 
time management, especially procrastination, could be one 
important mechanism for improving online learning. 


Clickstream data sets have provided rich resources for an- 
alyzing students’ time management behaviors. Procrasti- 
nation has been measured using the specific time points at 
which students take certain actions within an online course, 
such as accessing content pages, watching lectures, and sub- 
mitting quizzes. A common way to measure procrastination 
is to calculate the amount of time a student is engaged with 
the LMS prior to an important course deadline. Studies that 
use these types of measures as indicators of procrastination 
find that the indicators are negatively correlated with course 
outcomes [30]. In the context of studying planning 
behaviors, researchers have also developed measures of stu- 
dent regularity are in the timing and spacing of their course 
activities, and found that higher measures of regularity cor- 
relate with better performance [23] [3]. 


Motivated by these previous studies, we utilize clickstream 
data to further understand procrastination using two on- 
line classes offered at a large public university. These two 
classes were designed so that the students are expected to 
space out their studies on a daily basis, and to set weekly 
deadlines. In this paper, we investigate the use of proba- 
bilistic mixture modeling to analyze time-stamped logs of 
student activity in the context of these two online classes. 
The mixture model identifies different behavioral patterns 
in the data, where the patterns can be clearly identified as 
reflecting procrastinating and non-procrastinating behavior 
among the students. Moreover, we notice that while procras- 
tinating students may procrastinate frequently, some may 
also exhibit a mix of planning and procrastinating behaviors 
throughout the course. To capture these nuances, we con- 
struct a composite score, which incorporates both the overall 
degree of procrastination and the regularity of procrastinat- 
ing behaviors. This score captures behavioral differences of 
procrastinators, a notion which has been absent in prior re- 
search. The methodology we develop enables finer-grained 
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analysis of procrastination and its relationship with learn- 
ing outcomes, which can inform more effective instructional 
reforms in online learning. 


The primary contributions of this work are four-fold. 


e First, we develop a general data-driven method for 


identifying procrastination. This method analyzes counts 


of student activity and can work with any online course 
with periodic deadlines and that has corresponding 
time-stamped clickstream data. 


e Second, we validate this method using two online uni- 
versity classes, and identify two distinct behavioral 
patterns which can be used to measure an individual 
student’s degree of procrastination. 


Third, building off of prior measures of procrastina- 
tion, we investigate the regularity of procrastinating 
behaviors and incorporate this information into a com- 
posite score, providing a more detailed perspective on 
procrastination. 


e Fourth, for the two classes we analyze, we find that 
all of our measures of procrastination are highly cor- 
related with course outcomes, lending support to prior 
theories of self-regulated learning and procrastination 
while also providing new insights. 


2. RELATED WORK 
2.1 Self-Regulation, Procrastination, and 


Academic Success 

Self-regulated learning refers to the process of directing one’s 
own learning experience and these processes encom- 
pass several attitudes and behaviors. For instance, mod- 
els of self-regulated learning generally distinguish between 
motivational beliefs about learning, goal setting and plan- 
ning behaviors, specific learning strategies, and metacogni- 
tive monitoring processes [22]. While each of these facets 
play an important role in the learning process, research on 
online learning finds that students’ planning and time man- 
agement behaviors are important indicators of course success 
[30]. Procrastination behaviors, which refer to delaying 
coursework until major deadlines, reflect poor planning and 
time management. 


Several studies have focused on procrastinating as a ma- 
jor barrier that hinders students from succeeding in online 
courses [29]. Using online course analytic data, one re- 
cent study found that students who did not begin working 
on assignments until hours before a deadline received lower 
course grades when compared to students who began their 
work earlier [9]. Other studies have found similar results, 
where students who delay working on assignments are more 
likely to perform poorly [30]. These results confirm the 
undesirable nature of procrastination as well as the impor- 
tance of regular learning behaviors. 


Another extensive body of work has shown that students 
from underrepresented backgrounds, such as those who come 
from low-income households, or who are first to attend col- 
lege, are a greater risk for leaving STEM majors [7]. This 


problem may be additionally exacerbated in online course- 
work. There are many important factors that explain issues 
surrounding underrepresented student success, such as lack 
of mentoring, financial concerns, and feelings of exclusion 
[25]. With regard to self-regulatory behaviors such as pro- 
crastination, prior work has also shown an increased ten- 
dency for underrepresented groups to engage in more pro- 
crastination than the counterparts [24]. However, this study 
was not conducted in an educational context and the pro- 
crastination was measured subjectively using surveys. With 
this in mind, a side aim of our work is to explore the re- 
lationship between individual differences in procrastination 
(time management behavior, in general) and students’ exter- 
nal background characteristics, specifically for the students 
taking online courses. 


2.2 Measuring Procrastination and Regular- 
ity 

Measures of procrastination are relatively straightforward 
and similar across various learning environments. In the 
most common measures, researchers capture the time that 
students finish a certain task and calculate the difference be- 
tween this time and either the release time or the dead- 
line of the task. This type of measure has the merit 
of being very interpretable, but a limitation is that it only 
captures the average degree of procrastination without de- 
picting nuanced patterns in these behaviors. 


Regularity, on the other hand, is a higher-order concept that 
allows for different definitions. Accordingly, there has been 
a slightly larger pool of measures in the literature. Some 
studies define regular behaviors as repeating certain tempo- 
ral patterns in a cyclic manner, and apply methods from 
signal processing to model hidden frequencies within stu- 
dents’ behavioral streams 3]. Another popular way of 
operationalizing regularity is to relate regularity to changes 
of learner behaviors, and quantify the changes via measures 
of variation or explicit statistical modeling [21]. 
These different definitions are not exclusive and share many 
similar properties. 


Most of the existing studies regarding time management in 
online learning examine either procrastination or regularity, 
and those that investigate both treat them as independent 
features of student behaviors. Our work extend these stud- 
ies by understanding how regularity and procrastination are 
interrelated. 


2.3 Cluster Analysis and Mixture Modeling 

Clustering in general is a widely used technique in data anal- 
ysis for automated data-driven discovery of groups or clus- 
ters in data. In the context of analyzing education data, 
clustering algorithms have found broad application as a tech- 
nique for clustering of students into groups based on their 
behavioral patterns. For example, Toth et al. cluster 
students based on their problem-solving interaction patterns 
using the X-means algorithm (a variation of the well-known 
K-means clustering algorithm) for a better understanding 
of complex problem solving behaviors and identifying levels 
of problem solving proficiency. Ng, Liu, and Wang use 
survey scores of motivated strategies for learning question- 
naires to cluster students into multiple groups. The result- 
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ing groups obtained by hierarchical clustering with Ward’s 
method exhibit distinct learning profiles of motivational be- 
liefs and self-regulatory strategies. 


The clustering approach we follow in this paper is proba- 
bilistic model-based clustering [19]. In this framework, 
each cluster corresponds to a probability distribution (also 
known as a “component”) in a mixture model and the en- 
tities being clustered are assumed to have been generated 
by one of the component distributions. This probabilistic 
framework for clustering has a number of advantages over 
non-probabilistic techniques such as K-means clustering or 
hierarchical clustering. For example, as we describe later in 
the paper, the framework allows us to model count data in a 
natural manner using Poisson distributions as components in 
the mixture model, where each component (or cluster) rep- 
resents a different Poisson distribution over count outcomes. 
The Poisson mixture model has been applied to a number 
of different fields including marketing [4], finance |15], biol- 
ogy and bioinformatics (6| [1], document analysis |17|, and 
so on. However, to our knowledge, there has not been any 
prior work on the development of Poisson mixtures in an 
education context, particularly for the problem of clustering 
students based on their observed activity in online classes. 


3. METHODS 
3.1 Student Activity Counts 


For courses where time-stamped student-generated events 
are tracked via logs of clickstream data, we can count these 
events on a daily basis. Thus, we can get a set of daily activ- 
ity counts for each student throughout a course, where the 
activity can correspond to specific types of tasks of interest 
(such as video-watching, quiz submission, and so on) Figure 
[1] shows daily activity count data for 2 students from one of 
our course data sets. The data for each student is displayed 
as a matrix, where the grayscale indicates the number of 
tasks performed by each student on each day over the 5 
week duration of the course. This type of display is useful 
in terms of capturing the temporal aspect of when a student 
is engaged in a particular activity such as watching a lecture 
video or submitting a quiz. It also indicates that one of the 
students (on the right) may be procrastinating each week— 
we discuss these types of patterns in more detail later in the 
paper. 


We can also compute the sum over weeks to get the aggre- 
gated daily counts assuming that there is a structure in the 
course that repeats every week (which is the case for the two 
courses we study in this paper). Examples of the aggregated 
daily counts are shown in Figure[2]as bar plots, computed by 
aggregating across the weekly rows of data for each student 


in Figure 


3.2. Mixture Model with Gamma Priors 


In this section we discuss our use of a Poisson mixture model 
to cluster students based on their activity counts, focusing 
on the aggregated daily counts as in Figure [I] In terms of 
notation we let y; be the vector of aggregated daily counts 
for student i, where i = 1,...,N. The dimensionality D 
of each vector is the number of days (D = 6 in this case 
since Saturday and Sunday are collapsed into one). Thus, 
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Figure 1: Examples of student daily activity counts 
(specifically, the number of video watching tasks per 
day) displayed as a matrix of week x day counts. SS 
indicates Saturday and Sunday. 
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Figure 2: Aggregated daily task counts across weeks (y;) 
for the two students shown in Figure [I} 


our data consists of N students each with a D-dimensional 
vector of aggregated daily counts. 


To model this data we use a probabilistic mixture model 
with Poisson components. Let K be the number of compo- 
nents (or clusters) with an index k = 1,2,..., A. The unob- 
served latent variable z; takes values from the set {1,..., A} 
and corresponds to the latent component or cluster that 
student i is presumed to belong to. Each of the k com- 
ponents consists of a vector of Poisson rate parameters, 
Ak = [Aki,---;Akd,---;An6], where d from {1,...,6} rep- 
resents a specific day of the week. For example, one compo- 
nent could have very low values for all the Axa’s, for students 
with low daily activity, and another component could have 
high values for all the Axa’s, for students with high daily 
activity. 


When fitting our mixture model to data, we take a Bayesian 
approach and use Gamma prior distributions for the 
rate parameters Axa. The primary reason for doing this is 
to encourage the model to avoid degenerate solutions with a 
small component that has one or more rate parameters Axa 
at or near a value of 0. This can produce a high-likelihood 
solution but one that is not useful. In our experimental re- 
sults later in the paper we used hyperparameters of a = 1.1 
and 8 = 0.1 for the Gamma distribution. These hyperpa- 
rameter choices have the effect of making the Gamma prior 
behave like a step function, putting zero probability mass at 
Ara = 0,k = 1,...,K,d=1,...,6, and a relatively flat un- 
informative prior distribution over positive rate parameter 
values. Figure[3]depicts a graphical model representation of 
the Poisson mixture model with a Gamma prior on the A 
parameters for each component. 


The likelihood for the data y; for each student 7 under this 
mixture model can be written as 


P(ys|A) = D_ p(ysl2i = k, Ak) p(zi = k) (1) 


k=1 
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Figure 3: Graphical representation of the Poisson 
mixture model with Gamma prior. y; and A, are 6 
dimensional vectors. N is the number of students, 
and K is the number of mixture components. 


where p(z; = k) is the marginal mixing weight for each com- 
ponent, and each component distribution can be written as 


D 
P(yslzi =k, Ax) = | | plyealAca, x1 = k) (2) 


d=1 


assuming conditional independence of the daily counts yra 
given component k. Axa is the rate for day d for compo- 
nent k and each distribution p(yra|Ana, 21 = &) is a Poisson 
distribution. The prior distribution is defined as a product 
over independent Gamma priors, one for each Axa, each with 
parameters a = 1.1 and 6 = 0.1. 


3.3. Learning Parameters with the EM Algo- 


rithm 

To estimate the parameters Ax of our model we use the 
Expectation-Maximization (EM) algorithm, an iterative al- 
gorithm that is widely used in fitting mixture models to data 
[18]. More specifically, we use the EM algorithm to maxi- 
mize the product of the data likelihood times the prior (both 
defined above). This results in both (a) maximum a posteri- 
ori (MAP) parameter estimates for the Poisson components 
in the model, and (b) membership weights wi, that reflect 
the probability (under the fitted model) that each student 4 
belongs to component (or cluster) k. 


Each iteration of the EM algorithm consists of two steps, 
the E (expectation) step and the M (maximization) step. 
In the E-step, conditioned on some fixed (current) values of 
the parameters, the probability of membership w;, is com- 
puted for each component k = 1,...,K, for each student 
4=1,...,N. 


x p( Ys, 2 = k, Arla, 8) 
x p(yi|zi = k, An) p(Anl|a, 8)p(zi = k) (3) 


These membership weights are important in our later anal- 
yses, since they provide information of how likely it is that 
each data point i (in our case, student i) was generated by 
component k. In the M-step, conditioned on the set of mem- 
bership probabilities wiz, a point estimate of each parameter 
is estimated via MAP estimation. 


5 Dai Wik(Yi +o — 1) 
Ak = 


(4) 


Dy win +B) 
Ble: =k) = 2 (5) 


These MAP parameter estimates provide the input for the 
next E-step, and thus, the cycle of E and M-steps continue 
iteratively. 


The EM algorithm as a whole consists of randomly initializ- 
ing the parameters of the model, followed by repeated com- 
putation of pairs of E and M steps, until the log-likelihood 
is judged to have converged (i.e., when the improvement in 
log-likelihood from one iteration to the next is less than some 
small value €, or when the average membership probability 
value is not changing significantly from one iteration to the 
next). 


Python code for this EM algorithm is available online at 


https://github.com/jihyunp/student_poisson_mixture 


4. DATA SETS 


Two data sets from the same undergraduate online course 
were used in this study: one from summer 2016, and the 
other from summer 2017. Both summer courses were 5 weeks 
long. While each class was taught by two different instruc- 
tors, the class content, such as the lecture videos, resources, 
and assignments, were the same. In both classes, students 
were assigned 5 video lectures every week and each lecture 
video had a corresponding quiz. The instructors encouraged 
students to watch one lecture video and complete the corre- 
sponding quiz each day, from Monday through Friday. 


Although students were encouraged to follow this schedule, 
the actual deadline for watching the 5 lecture videos and 
completing the quizzes was on Fridays at midnight. While 
this structure gave students freedom to watch the lecture 
videos when they wanted, this flexibility also allowed them 
to procrastinate. 


Most of the students’ activities were recorded through the 
Canvas Learning Management System (LMS). These activ- 
ities included downloading course content, watching lecture 
videos, taking online quizzes, submitting assignments, etc. 
Every time a student clicked on a URL within the Can- 
vas system, the click event was logged with the student ID, 
URL, and time-stamp. The clickstream data was processed 
so that it only focused on the activities of daily tasks, re- 
sulting in daily activity counts, as mentioned in the previous 
section (F igure [I] and Figure [2h. Only one event per task 
was counted and thus the sum of the matrix for each stu- 
dent was 25 or less (for 5 video lectures x 5 weeks). We 
chose to count only the first attempt (first click event) for 
each task. 


In addition to the clickstream data, student demographic 
data was available through the university’s institutional re- 
search office. It included both demographic information 
(gender, ethnicity, first generation status, low income sta- 
tus, and full-time status) and prior academic achievement 
(total sATf] score). Some students did not agree to provide 
this demographic data, although most did. For this rea- 
son, our later analyses based on demographic information 
are based on the subset of students who agreed to share this 
information. 


‘A standardized test widely used for college admissions in 
the United States. 
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Figure 4: Grade distributions of students in 2016 
class (left) and in 2017 class (right). Two classes 
show very different grade distributions. Almost half 
of the students received an A for the class in 2016, 
whereas more students got lower grades in 2017. 


Although both classes used the same materials and imple- 
mented the same deadlines, there were some notable dif- 
ferences in how the the click events were recorded, as well 
as how each instructor structured the course. We describe 
these differences in the following sections. 


4.1 Class in 2016 


Online lectures and daily quizzes were offered outside the 
Canvas LMS for this class. Each lecture video was embedded 
on a separate web page on the server that we had access to, 
and the links to the web pages were provided via the Canvas 
weekly module. 


Logs for the daily quiz attempts were not accessible, so in- 
stead we used the first “video clicks,” which are from the logs 
of HTTP GET requests of the video embedded web pages. 
For each student, we matched the IP addresses of the video 
logs (from the server) with the IPs recorded on the Canvas 
LMS. 


After removing 4 students with very low activity (0 or 1 
video clicks in total) there were 172 students with activity 
counts available for analysis. More than 90% of the stu- 
dents received a passing grade, and half of the students re- 
ceived an A (F igure [4p. Completing the daily tasks (watch- 
ing videos and solving quizzes) counted as 15% towards the 
overall grade for each student. 


4.2 Class in 2017 


The video click logs for this class were not available since the 
videos were uploaded on a third-party server. However, the 
daily quizzes that students took after watching the lecture 
videos were recorded through the Canvas system, and we 
were able to obtain students’ quiz submission time-stamps 
via the corresponding clickstream data. Therefore, for this 
class we focused on the first clicks for daily “quizzes.” Note 
that this is different from the 2016 class data, which used 
the first clicks for each video-watching event. 


There were 145 students in the class—we used data for 140 
students after dropping 5 students with very low activity 
(as with the 2016 class). As previously noted, a different 
instructor taught the class in 2017 than in 2016. The in- 
structor for the 2017 class changed the contribution to 8% 
of the total grade for watching and completing the lecture 


(a) Class in 2016 


| ial 


TOTAL NUMBER OF 
FIRST VIDEO CLICKS 
ON EACH DAY OF WEEK 


ss M T W TF ss M T W TF 
(b) Class in 2017 


TOTAL NUMBER OF 
FIRST QUIZ ATTEMPTS 
ON EACH DAY OF WEEK 


Figure 5: Poisson mixture component means (A,’s) 
from modeling aggregated daily task counts (y;) for 
the class in 2016 (upper) and 2017 (lower). 


videos, significantly less than in the 2016 class (15%). The 
grade distribution of the 2017 class in Figure [4] is also sig- 
nificantly different to that in 2016—there are significantly 
fewer students who received A’s or B’s in 2017 compared to 
the 2016 class. 


5. PROCRASTINATION AS A MIXTURE 
COMPONENT 


Below, we present and discuss the results of fitting a two- 
component (C = 2) Poisson mixture model to the aggre- 
gated daily task counts for the two classes described in sec- 
tion We also explored models with more components, 
K = 3,4,..., but found that the K = 2 model broadly cap- 
tured the primary modes of student behavior and that higher 
values of kK tended to split the two main modes into further 
subgroups without providing any significant additional in- 
sight. 


Figure |5| shows the expected number of counts per group, 
ie., the rate parameters, Ax’s. The two group-dependent 
rate patterns across the days of the week, for both 2016 and 
2017, show two very distinct behavioral patterns. One of 
the mixture components has a very high rate on Friday and 
low rates on the other days of the week. The other compo- 
nent has low and relatively flat rates from Monday to Fri- 
day. Considering the fact that the deadline for daily tasks in 
these courses is on Fridays, these two patterns clearly reflect 
two different types of student behaviors: procrastination and 
non-procrastination. 


5.1 Characteristics of the Two Behavioral 


Groups 
We can threshold the membership weights at 0.5 to classify 
each student 7 = 1,...,N into one of the two groups, i.e., 


if wi1 > 0.5 then student 7 is assigned to the procrastina- 
tion group (where k = 1 corresponds to the procrastination 
group). About 36-37% of the students were assigned to the 
procrastination group in each of the two years. 
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Figure 6: Aggregated daily task counts shown along with the membership weights. Each row represents a 
student, and the students are sorted by the membership weight w;;. The left figure is for the class in 2016, 


and the right figure is for the class in 2017. 
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Figure 7: Probability of receiving each grade given 
that the student is in the procrastination group or 
in the non-procrastination group in 2016 (left) and in 
2017 (right). 


The two plots in Figure [6] illustrate the students’ week- 
aggregated activities along with the students’ membership 
weights. Each row in each plot represents a student and 
the wider matrix plot shows the aggregated daily counts, 
sorted by their membership weight wii. The values in the 
matrix range from 0 to 25 and a darker color means that 
there are more task activities on that day of the week. The 
two plots from different years look almost identical and they 
clearly show the two types of behavior. The students (rows) 
at the bottom of each plot have more counts (darker colors) 
on Fridays and belong to the procrastination group. There 
is also a small group of students at the top of both plots 
who tend to be more active over the weekend. The size of 
this group of students is relatively small and their behavior 
pattern is effectively that of non-procrastinators since they 
are the “early birds” who check out the lecture videos or the 
quizzes early in the week. 


The membership weights are shown on the narrower bar plot 
(left of each year’s plot), where a darker color represents a 
higher membership weight of belonging to the procrastina- 
tion group (with a weight close to 1). We can observe that 
there is a relatively small amount of grey area in the bar 
plot (for both years), which means that the majority of the 
students have a very high probability of being assigned to 
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Figure 8: Distribution of wi, in different grade 
groups of class in 2016 (left) and in 2017 (right). 
H-statistic comes from a Kruskal-Wallis test. (wii: 
membership weight on the procrastinating group) 


one group or the other. 


5.2. Association between Behaviors and Grades 
We can further analyze the relationship between the two 
different behavioral groups and the grades. We show the 
grade distribution in each group in Figure Results from 
the two classes are shown side by side. It is obvious from 
the figure that the non-procrastinators tend to get signifi- 
cantly more A grades than the procrastinators, whereas the 
procrastinators get more C, D, and F’s. Even though the 
overall grade distributions were quite different in the two 
classes (see Figure |4), we find a strong correlation between 
the behavioral groups and course outcomes. In both classes, 
the non-procrastinating students are about three times more 
likely to get an A grade than the procrastinating students. 
These probabilities were significant at the 0.01 level using a 
chi-squared test. 


We can further analyze the relationship between procrasti- 
nation behavior and grade outcomes by grouping students 
by their grade (rather than by the behavioral group) and 
looking at the patterns of behavior for each grade group. 


As we saw in Figure[4] a majority of the students got a pass- 
ing grade in 2016. The number of students who received A, 
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Figure 9: The number of task counts per day, for 
each of the 5 weeks, averaged over the students in 
each grade group. Left: students who received A’s, 
middle: students who received B’s, right: students 
who received a C, D, or F. 


B, and the others (C, D, or F) were 84, 66, and 22, respec- 
tively. The left boxplot in Figure[8]informs that “A students” 
have very low membership weight (wi1) values, but the “C, 
D, F students” have very high membership weight values. 
This can be interpreted as saying that the students who re- 
ceived lower grades (C, D, or F) have higher probabilities of 
being procrastinators. 


We can see the similar result for the class in 2017 from the 
right side of the plot in Figure There were 27, 37, 49 
students in each of the grade groups (there were 27 students 
whose grade information was unavailable). The broader dis- 
tribution of weights in the C, D, F group may be due to the 
fact that there were many more students with lower grades 
than higher grades in this year of the course. 


The association between behavior and grade outcome is also 
clearly visible in the raw data, i.e., the task activity counts, 
for both years. Figure[9| clearly illustrates the behavior pat- 
terns for students with different grades. We can see a very 
dark color on Fridays on the matrices on the right side (stu- 
dents who received C, D, or F grades), and more evenly 
distributed colors on the left matrix plots, which shows the 
activities of students who received A grades. 


6. REGULARITY OF PROCRASTINATION 


In the previous section we showed that Poisson mixture 
modeling can help to unveil two latent types of students: 
procrastinators and non-procrastinators. Because these re- 
sults are based on modeling aggregated daily activity counts 
across multiple weeks, they do not shed light on how stu- 
dents might change their procrastinating behavior over time 
during different stages of a course. For example, a student 
who is generally a procrastinator might only procrastinate 
every other week, while a non-procrastinator might postpone 
studying during some week. To gain insights into these nu- 
ances, we investigate the regularity of procrastination in this 
section. 
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(b) Class in 2017 
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Figure 10: Poisson mixture component means (A;’s) 
from modeling individual week of daily task counts 
(yi;) for the class in 2016 (upper) and 2017 (lower). 
The number of first clicks on any lecture video in 
2016, and the number of first attempts on any quiz 
in 2017, are modeled. 


6.1 Regularity across Weeks 

We focus here on inter-week regularity, which is defined as 
the extent that students repeat their behavior across dif- 
ferent weeks. We use the same Poisson mixture modeling 
methodology described earlier in the paper except that we 
model each individual week of daily activity counts for each 
student rather than aggregating across weeks. The result- 
ing mixture components are similar to the aggregated case 
in that there are two distinct weekly behaviors, procrastina- 
tion and non-procrastination (see Figure [10). Each week of 
a student’s behavior is modeled as being generated by one 
of the two components in the model, and we can estimate 
the membership weight of belonging to the procrastination 
group (or component) for each week for each student, i.e., 
wij1 for student 7 = 1,...,N and week j = 1,...,M where 
M = 5 is the number of weeks|-] 


To quantify student 7’s regularity, we use the standard de- 
viation of the procrastination weights across weeks: 


SD; = Ce 7 3 (wij — win}*) (6) 


j=l 


where wij1 and W;.1 represent the student-week membership 
weights for the procrastination component in week j, and the 
average of those weights across the M weeks, respectively. 


?We could also use the non-aggregated weekly daily activity 
counts for the earlier group analyses in Section] Instead of 
the membership weights w,;, for student 7, the mean value of 
the M membership weights (W;.1) could be used for thresh- 
olding. This would allow us to use the same analyses in 
pouomene by fitting a single mixture model. We inves- 
tigated this and found the results were almost identical to 
those reported in the paper. Given this, for the investigation 
of regularity we used weekly activity counts to see changes 
in weekly behavior, and for overall clustering (Section|5) we 
used total aggregate counts for ease of interpretation. 
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Figure 11: Distribution of SD; in different grade 
groups of class in 2016 (left) and in 2017 (right). 
H-statistic comes from a Kruskal-Wallis test. (SD;j: 
inter-week standard deviation of w;;1, membership 
weight on the procrastination group in week 7) 
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Figure 12: Distribution of SD; in two behavioral 
groups (see Figure [6) of class in 2016 (left) and 
in 2017 (right). Within each subgraph, the pro- 
crastination group is indexed as 1, while the non- 
procrastination group is indexed as 2. (SDj;: inter- 
week standard deviation of w;;1, membership weight 
on procrastination group in week 7) 


By definition, a higher value for SD; signifies more volatile 
behavioral patterns. 


In light of prior research, regularity is strongly correlated to 
performance [3]. We plot the distribution of SD;’s within 
three grade groups in F igure[I]] Consistent with prior find- 
ings, students with better grades in general have lower levels 
of SD;, hence are more regular learners. More formally, we 
perform a Kruskal-Wallis test within each class, with results 
reported above the graph. In both years, the three groups 
have significantly different SD, distributions. 


6.2 Incorporating Regularity and Procrastina- 
tion 
In previous sections, wi: and SD; capture different dimen- 
sions of procrastinating behavior, and their interaction is 
worth discussing further. For one thing, procrastinators and 
non-procrastinators may have different levels of regularity. 
We compare the distribution of S'D; within each behavioral 
group (assigned identically as in F igure |6) and plot the re- 
sults in Figure Common to both classes, procrastina- 
tors are centered around 0.4, while non-procrastinators on 
average have very small values below 0.1. We also calcu- 
late Pearson’s correlation coefficient between w;i (continu- 
ous membership weights before hard group assignments, as 
defined in Section [5) and SD;j, resulting in values of 0.675 
for 2016 and 0.590 for 2017, both statistically significant 
at the 0.001 level. From these results, we can conclude 
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Figure 13: Number of daily activity counts for four 
prototypical students in the 2016 class. (wii: aggre- 
gated membership weight on procrastination group; 
SD;: inter-week standard deviation of w;;1, member- 
ship weight on procrastination group in week 7; TM;: 
Time Management Score) 
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Figure 14: Distribution of Time Management Score 
(TM;) in different grade groups of class in 2016 
(left) and in 2017 (right). H-statistic comes from 
a Kruskal-Wallis test. 


that non-procrastinating students are also more likely to 
stay consistent throughout the course, while procrastinators 
jump between spacing out their studies and postponing ev- 
erything until the last day. On the other hand, the rela- 
tionship between regularity and academic performance may 
substantially vary depending on how much a student is a 
procrastinator. Procrastinators who put off studying as a 
habit (with high regularity) may be more at-risk than those 
who occasionally jump to a spaced-out pattern, while this 
is the opposite for non-procrastinating students. To incor- 
porate this asymmetry, we attempt to define a single index 
built upon wii and SD;. As these two measures are both 
conceptually related to time management abilities, we name 
the index to be the Time Management Score (J'M;). To 
reflect their interaction, we multiply variations of wi, and 
SD; for each student i. Since wii and SD; are both nega- 
tively correlated with outcome, we use the negative of their 
values in the index to allow for more natural interpretation. 
Moreover, because the score should be weighted in oppo- 
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site directions depending on the student’s behavioral group, 
we made variations to wi1 so that the most procrastinat- 
ing student with the same degree of regularity would have 
the smallest score. Taking all of the above into account, we 
define TM; as follows: 


TM; = (1 = wit) (1 — SD;) + [—wii (1 — SD;)} 


= (1 — 2wa1) (1— SDi) (7) 


To evaluate the validity of this index, we examine whether 
its properties are aligned with theoretical assumptions. As 
discussed above, from the perspective of academic success, 
higher regularity is a negative behavioral feature for procras- 
tinators but is a positive feature for non-procrastinators. In 
this context, it is natural to investigate how regularity and 
procrastination affects the value of TM;, and how this value 
relates to desirable and undesirable outcomes. 


From Equation (7), we know that the value of TM; is pos- 
itive or negative depending on whether wi; is greater than 
0.5 or not. Thus, wi1 = 0.5 is the watershed of whether SD; 
positively or negatively contributes to T7M;. Given that our 
threshold for hard group assignment in Section []is also 0.5, 
the interpretation is straightforward: higher regularity leads 
to higher TM; within the procrastination group, and it is 
more so for “purer” procrastinators; the opposite story can 
be told within the non-procrastination group. 


For an intuitive examination, we choose four prototypical 
students with different levels of procrastination and regu- 
larity from the classs in 2016, and plot the daily counts of 
their first video clicks in Figure along with their wi1, 
SD, and TM;. As we would expect, non-procrastination 
with high regularity (upper-left), the most desirable pattern, 
has TM; = 1, the maximum value possible in our context. 
By contrast, the regular procrastinator (lower-left) gets the 
minimum value of TM; = —1. The remaining two students 
with similarly low regularity have TM; values between the 
two extremes, but are respectively closer to the one that be- 
longs to the same behavioral group. In a word, these visual 
patterns further validate the construction of 7/M;, which 
more precisely measures the degree of procrastination by 
incorporating regularity information. 


To determine if 7M; captures the desirability of certain 
procrastinating patterns, we probe into the relationship be- 
tween this index and course outcomes. Similar to what we 
did earlier with wi; and SD; individually, we plot the dis- 
tribution of TM; within three grade groups. As shown in 
Figure [14] there exists a positive relationship between TM; 
and performance, which is statistically significant under a 
Kruskal-Wallis test. The 7'M; score incorporates two mea- 
sures (w;; and SD;) and amplifies the information that is 
potentially predictive of performance, providing a more nu- 
anced view of procrastination. 


7. RELATIONSHIP WITH STUDENT 
BACKGROUND 


Having explored the fine-grained differences in students’ pro- 
crastinating behaviors and their relationship with outcomes, 
we want to further examine if these variations can be dis- 
criminated by students’ background characteristics. The 
goal of this analysis is to understand whether there exists 


Table 1: Relationship between demographic vari- 
ables and procrastination/regularity measures for 
the 2016 class 


(a) Behavioral group assignment (binary) 


Demographics | N Test p-value 

FirstGen 144 2g 0.566 

LowInc 151) = | ere 

SAT 147 | K-W test | 0.238 

(b) SD and TM (continuous) 
Demographics | N Test SD p-val | TM p-val 

FirstGen 144 0.884 0.954 
Lowine . | 151 | |. .g.175 0.294 
SAT 147 | Pearson’s r 0.118 0.363 


Table 2: Relationship between demographic vari- 
ables and procrastination/regularity measures for 
the 2017 class 


(a) Behavioral group assignment (binary) 


Demographics | N Test p-value 

FirstGen 120 rae 0.218 

LowIne ioe) ee 0.955 

SAT 125 | K-W test | 0.802 

(b) SD and TM (continuous) 
Demographics | N Test SD p-val | TM p-val 

FirstGen 120 0.136 0.897 
Lowine | 198 | HW test | o 754 0.973 
SAT 125 | Pearson’s r 0.505 0.820 


a potential risk factor among underrepresented students, or 
if instead, the behavioral differences we observe are more 
individual-level in nature. We also sought to explore whether 
prior academic achievement could explain differences in pro- 
crastinating behaviors. 


From a comprehensive list of demographic variables, we choose 
three that are of general interest in education research: Low 
Income Status, First Generation and Total SAT Score. The 
first two binary variables represent a student’s social-economic 
status, and the last continuous variable is a proxy for prior 
academic achievement. 


We separately test the relationships between these three 
variables and three measures of procrastination and regu- 
larity in previous sections: behavioral group assignment (as 
in Section [5.1p, SD and TM (as in Section [6p. The specific 
statistical tests we use and their results are reported in Ta- 
ble[I] for the class in 2016, and Table [2] for the class in 2017. 
Because the demographic information contains missing val- 
ues, we only include students who have relevant information 
in each of the tests (the number of students, N, is reported 
in the tables). 


The results show that for both classes none of these de- 
mographic variables have any significant relationship with 
procrastination and/or regularity. This suggests that fail- 
ures in time management may arise more from students’ in- 
herent factors than specific background characteristics, and 
that effective instructional interventions are less likely to 
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be hampered by students’ underrepresented backgrounds. 
However, due to the limited class sizes, this inference still 
needs to be further explored at scale. 


8. CONCLUSIONS 


In this paper, we introduce a data-driven methodology for 


characterizing student procrastination in online courses. Based 


on Poisson mixture modeling, the proposed approach can be 
applied to courses where tasks with clear deadlines are regu- 
larly assigned and students’ timestamped activites related to 
those tasks are recorded. In our experiments with two under- 
graduate online classes, this method identifies two distinct 
patterns in students’ weekly planning behavior, which can 
be further utilized to measure procrastination. This mea- 
sure is found to be strongly correlated with course outcomes 
for both classes. In addition, our proposed Time Manage- 
ment Score (7'M) is able to quantify students’ overall time 
management skills by combining overall degree of procras- 
tination with the regularity of the behavior. Interestingly, 
while TM is a strong predictor of course outcomes, it is not 
significantly related to students’ demographics or prior aca- 
demic achievement. These results suggest that, as a whole, 
procrastination behaviors seem to be more of an inherent 
characteristic. 


These types of clickstream data and analyses allow for rich 
complements to other types of educational research. For 
example, the proposed behavioral measures of time man- 
agement can be combined with survey data to examine how 
accurate students’ perceptions of their skills are, and to iden- 
tify students who might be especially prone to benefit from 
support. From the practical perspective, these data-driven 
approaches can be incorporated into learning management 
systems and work in real time. This would potentially facil- 
itate automated assessment and intervention regarding time 
management skills. 


There are also a number of potentially useful extensions to 
the methodological approach proposed here. For example, 
the mixture components in the two classes that we analyzed 
are straightforward to interpret with regard to procrasti- 
nation, but this might not be the case for different course 
designs and structures. In these broader scenarios, it may be 
useful to incorporate informative Gamma prior distributions 
into the mixture model, with, for instance, three prior com- 
ponents for procrastination behavior, non-procrastination 
behavior, and mixed behavior respectively. 
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